Information processing apparatus and method for controlling information processing apparatus

ABSTRACT

An information processing apparatus includes a display unit, a first detection unit that detects a position of an object located in a vertical direction with respect to the display unit and that interacts with the display unit in a contactless manner, wherein detecting the position results in detecting an instruction associated with the at least one input object that is displayed on the display unit and corresponds to the position of the object, a second detection unit configured to detect confirmation of the instruction detected by the first detection unit, and an input unit configured to input data corresponding to the at least one input object in a case where the second detection unit detects confirmation of the instruction associated with the at least one input object in a state where the first detection unit detects the instruction associated with the at least one input object.

BACKGROUND Field

The present disclosure relates to an information processing apparatusand a method for controlling the information processing apparatus.

Description of the Related Art

One result of the free movement of persons and goods in recent years hasresulted in an outbreak of a new virus and variants of the virusinstantly spreading across the world resulting in a significant numberof infected individuals. Many infectious diseases are known to infecthumans through droplet infection. Specifically, a person is infected byinhaling viruses contained in droplets expelled from other infectedindividuals. Research results have indicated that certain viruses whenlocated on glass and/or plastic surfaces can remain infectious forseveral days.

This phenomenon has increased the demand for various methods ofcontactless interaction to avoid as much contact with surfaces harboringviruses as possible. One such surface includes those used to operateelectronic devices used by an unspecified number of persons. Forexample, a multi-function peripheral (MFP), which is an image formingapparatus, is a single apparatus used by multiple users and includes atouch panel display for receiving operations from these users.

Japanese Patent Application Laid-Open No. 2016-62410 discusses an MFPwith a touch panel display that detects a contactless input. The touchpanel display detects whether a user's finger that is indicated by X-,Y-, and Z-coordinates of a space above a touch panel is relatively closeor far. Since the contactless input is confirmed without a physicaltouch, the user does not experience, for example, click feeling. Thiscan result in the user making multiple inputs, processing proceeding tothe next step without input, etc., which can require the user tore-start the desired processing.

Japanese Patent Application Laid-Open No. 2016-62410 is seen to discussan operation of touching the panel to confirm input. Japanese PatentApplication Laid-Open No. 2016-62410 discusses a technology fordetecting a hover position of a user's finger to prevent input errors incontactless key input. Proximity is also detected, and an auxiliarydisplay is provided at a nearby position shifted from the hover positionbased on the proximity so that an input target key is not hidden by theuser's finger to prevent input errors. Japanese Patent ApplicationLaid-Open No. 2014-92988 is also seen to discuss an operation oftouching the panel to confirm input.

Japanese Patent Application Laid-Open No. 2017-21827 discusses anoperation of confirming contactless input in a case where a user'sfinger hovers a predetermined time or longer at a hover position.

Whether contactless input is confirmed can be difficult for a user todetermine due to an absence of a click feeling because a display screenand a user's finger are separated from each other and the finger doesnot touch an object. In confirming input based on a predetermined time,whether the input is confirmed can also be difficult to determinebecause it is difficult for the user to synchronize the input with adetermination timing of the apparatus. Thus, erroneous user input canoccur. Specifically, processing can proceed to a next operation withoutconfirming input and re-starting the processing may become necessary, orinput can be redundantly confirmed.

SUMMARY

The present disclosure is directed to a method for clearly confirmingcontactless input.

According to an aspect of the present disclosure, an informationprocessing apparatus includes a display unit configured to display atleast one input object corresponding to data input, a first detectionunit configured to detect a position of an object located in a verticaldirection with respect to the display unit and that interacts with thedisplay unit in a contactless manner, wherein detecting the positionresults in detecting an instruction associated with the at least oneinput object that is displayed on the display unit and corresponds tothe position of the object, a second detection unit configured to detectconfirmation of the instruction detected by the first detection unit,and an input unit configured to input data corresponding to the at leastone input object in a case where the second detection unit detectsconfirmation of the instruction associated with the at least one inputobject in a state where the first detection unit detects the instructionassociated with the at least one input object.

Further features will become apparent from the following description ofexemplary embodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a front external view illustrating an image forming apparatus.

FIG. 2 is a top view illustrating the image forming apparatus.

FIG. 3 is a schematic block diagram illustrating the image formingapparatus.

FIG. 4 is a block diagram illustrating a controller unit.

FIG. 5 is a block diagram illustrating an operation unit.

FIG. 6 is a block diagram illustrating an infrared contactless inputunit.

FIG. 7 is a schematic view illustrating an infrared operation unit.

FIG. 8 is a view illustrating contactless input via the infraredoperation unit.

FIG. 9 is a block diagram illustrating a capacitive operation unit.

FIG. 10 is a schematic view illustrating an operation panel of thecapacitive operation unit.

FIG. 11 is a flowchart illustrating an operation of detecting user inputby the operation unit.

FIG. 12 is a flowchart illustrating an operation of confirming userinput by the controller unit.

FIGS. 13A to 13L illustrate facsimile destination input operationscreens.

FIG. 14 is a front external view illustrating an image formingapparatus.

FIG. 15 is a top view illustrating the image forming apparatus.

FIG. 16 is a schematic block diagram illustrating the image formingapparatus.

FIG. 17 is a block diagram illustrating a controller unit.

FIG. 18 is a flowchart illustrating an operation of confirming userinput by a foot switch of the controller unit.

FIG. 19 is a flowchart illustrating an operation of confirming userinput by voice recognition by the controller unit.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments will be described in detail below with referenceto the attached drawings. The exemplary embodiments described below arenot intended to limit the scope of the enclosed claims, and not allcombinations of features described in the exemplary embodiments arealways essential to implementing a solution of the present disclosure.While an image processing apparatus will be described in the exemplaryembodiments as an example of an information processing apparatus, theinformation processing apparatus is not limited to the image processingapparatus.

FIG. 1 illustrates an external view of the image forming apparatus, FIG.2 illustrates a top view of an image forming apparatus 10, and FIG. 3illustrates a block diagram of the image forming apparatus 10 accordingto a first exemplary embodiment. The image forming apparatus 10 in thepresent embodiment is a multi-function peripheral (MFP) device with aplurality of functions such as a print function, a scanner function, anda copy function.

The image forming apparatus 10 includes an operation unit 200, acontroller unit 100, a printer unit 120, a scanner unit 130, a powerunit 300, a power switch 148, and a facsimile unit 160. The imageforming apparatus 10 is connected to a network 118.

The controller unit 100 controls the components of the image formingapparatus 10. A user performs various operations on the image formingapparatus 10 via the operation unit 200. Details of the controller unit100 and the operation unit 200 will be described below.

The power unit 300 provides power to the components in the image formingapparatus 10. When the power is off, alternating-current (AC) power isinsulated by the power switch 148, and when the power switch 148 isturned on, AC power is provided, and the power unit 300 generatesdirect-current (DC) power. The generated DC power is provided to thecomponents in the image forming apparatus 10.

The scanner unit 130 reads a document and generates monochrome binaryimage data or color multi-valued image data. The printer unit 120 printsthe image data generated by the scanner unit 130 on a sheet. Thefacsimile unit 160 can transmit and receive digital images to and from atelephone line 163.

The controller unit 100 will now be described with reference to FIG. 4 .

A central processing unit (CPU) 105 executes a software program storedin a random access memory (RAM) 104 and controls the image formingapparatus 10.

A read-only memory (ROM) 103 stores, for example, a program forexecution relating to activation of the controller unit 100 and fixedparameters.

The RAM 104 is used to store programs and temporary data by the CPU 105in controlling the image forming apparatus 10. The programs andtemporary data stored in the RAM 104 are read from the ROM 103 or astorage 502 described below.

A printer control unit 122 receives settings information viacommunication with the CPU 105 and controls operations of the printerunit 120 based on the settings information. The settings information isset by the user via a printer interface (printer I/F) 121 in performingprint processing.

A scanner control unit 132 receives settings information viacommunication with the CPU 105 and controls operations of the scannerunit 130 based on the settings information. The settings information isset by the user via a scanner interface (scanner I/F) 131 in performingscanner processing.

A network interface (network I/F) 106 transmits and receives data to andfrom the network 118. Specifically, the network I/F 106 receives datatransmitted from the network 118 and transmits image data obtained byscanning by the scanner unit 130 and image data stored in the storage502 to predetermined destinations via the network 118.

A facsimile interface (facsimile I/F) 161 transmits and receives digitalimage to and from the telephone line 163 via the facsimile unit 160. Forexample, the facsimile I/F 161 receives image data transmitted from thetelephone line 163 via the facsimile unit 160. In another example, thefacsimile I/F 161 transmits image data obtained by a scanning operationby the scanner unit 130 and image data stored in the storage 502 topredetermined destinations via the facsimile unit 160 and the telephoneline 163.

The storage 502 as a main storage stores programs to be executed by theCPU 105, program management tables, and various types of data. Theprograms to be executed include a boot program that is executed by theCPU 105 to activate an operating system (OS) in activating the imageforming apparatus 10.

Examples of the storage 502 include a hard disk drive (HDD), asolid-state drive (SSD), an embedded multimedia card (eMMC), a NANDflash memory, and a NOR flash memory. The controller unit 100 isconnected to the operation unit 200 via an operation unit interface(operation unit I/F) 199.

The operation unit 200 will now be described with reference to the blockdiagram in FIG. 5 . The operation unit 200 includes a control board 201,a display device 209, and a contactless touch panel 210.

An image processing unit 203 generates image data for display on thedisplay device 209 and transmits the generated image data to the displaydevice 209 via a display device interface (display device I/F) 206. Atouch panel control unit 204 acquires coordinates data from thecontactless touch panel 210 and communicates with the controller unit100 via a controller interface (controller I/F) 205. The controller I/F205 is connected to the controller unit 100 and is connected to thecomponents of the control board 201 via a system bus 208. Thecontactless touch panel 210 is superimposed on the display device 209.In a case where the user selects an icon displayed on the display device209, the contactless touch panel 210 receives the input.

An example of the contactless touch panel 210 using an infrared methodwill be described with reference to FIG. 6 . The contactless touch panel210 includes a control unit 211, light emitting elements 158 and 168,and light receiving elements 159 and 169. The control unit 211 includesa detection unit 212, a determination unit 213, and a storage unit 214.

The detection unit 212 transmits a result of detection of intensities oflight received by the light receiving elements 159 and 169 to thedetermination unit 213. The determination unit 213 stores theintensities of light received by the light receiving elements 159 and169 and transmitted from the detection unit 212 in the storage unit 214.The determination unit 213 transmits information about coordinates ofthe light receiving elements 159 and 169 with a change in received lightintensities, threshold values, and reference value data to the touchpanel control unit 204 of the control board 201 via a touch panel I/F207. The light emitting elements 158 and 168 consist of, for example,infrared light emitting diodes (infrared LEDs). The plurality of lightemitting elements 158 and 168 is located near the display device 209.

The light emitting elements 158 and 168 emit light based on instructionsfrom the detection unit 212. The light emitted from the light emittingelements 158 and 168 is diffracted, shielded, or reflected by a user'sfinger. The light receiving elements 159 and 169 consist of, forexample, photodiodes. The plurality of light receiving elements 159 and169 located near the display device 209 transmits received lightintensities to the detection unit 212.

FIG. 7 is a view illustrating the operation unit 200. The display device209 of the operation unit 200 includes a function of the contactlesstouch panel 210 that uses an infrared method. The plurality of lightemitting elements 158 and 168, such as infrared LEDs, and the pluralityof light receiving elements 159 and 169, such as phototransistors, ofthe contactless touch panel 210 are located opposite each other along anouter frame of the operation unit 200. There is no specific limit to thenumber of light emitting elements 158 and 168 and the number of lightreceiving elements 159 and 169 that are provided, and any numbers thatenables detection of an operation input and confirmation input by auser's finger is applicable.

In the present exemplary embodiment, twenty-two light emitting elements158, twenty-two light receiving elements 159, thirty light emittingelements 168, and thirty light receiving elements 169 are provided. Acontactless touch detection region of the contactless touch panel 210 iswithin the display device 209. In a case where the contactless touchpanel 210 is operating and invisible light such as infrared light fromthe light emitting element 158 to the light receiving element 159 andinvisible light such as infrared light from the light emitting element168 to the light receiving element 169 are blocked by a user's finger,the detection unit 212 detects coordinates of a blocked position. Forexample, as illustrated in FIG. 8 , the user's finger is insertedvertically to the contactless touch panel 210 and blocks infrared lightfrom the light emitting elements 158 and 168. This decreases theintensities of the light received by the light receiving elements 169-13to 169-18 and the light receiving elements 159-8 to 159-11, and it isdetermined that an “operation key (3) 800” is selected.

Next, capacitive contactless input will be described with reference toFIG. 9 . FIG. 9 is a diagram illustrating an example of a capacitivetouch panel 215 that is included in the operation unit 200.

The touch panel control unit 204 acquires the coordinates data from thecapacitive touch panel 215 and communicates with the controller unit 100via the controller I/F 205. The capacitive touch panel 215 issuperimposed on the display device 209. In a case where an operation keydisplayed on the display device 209 is selected by the user, thecapacitive touch panel 215 receives the input. The capacitive touchpanel 215 includes a control unit 170 and a sensor unit 174. The controlunit 170 includes a detection unit 171, a determination unit 172, and astorage unit 173. The sensor unit 174 includes a driving unit 175, adetection data generation unit 176, and an operation panel 177. Theoperation panel 177 is used as a user interface for input.

The operation panel 177 is illustrated in FIG. 10 . The operation panel177 includes a plurality of electrodes Ex 178 extending in a firstdirection (e.g., X-axis direction) and a plurality of electrodes Ey 179extending in a second direction (e.g., Y-axis direction) perpendicularto the first direction. The electrodes Ex 178 and the electrodes Ey 179being insulated from each other intersect with each other, and acapacitive sensor element 180 is formed near each intersection point.The electrodes Ex 178 and the electrodes Ey 179 are not limited tostripe shapes and can be any shape that includes intersection points,such as diamond patterns (rhombic patterns), that enable practice of thepresent disclosure.

Turning back to FIG. 9 , the driving unit 175 applies a driving voltageto each sensor element 180. The driving unit 175, for example,sequentially selects the plurality of electrodes Ex 178 and applies aperiodically changing voltage to the selected electrodes Ey 179 based oncontrol by the detection unit 171. The application changes electricpotentials of the sensor elements 180, and charges and discharges occur.The electrodes Ey 179 detect electrostatic capacitances of the sensorelements 180 by detecting an amount of electric charges. The electrodesEy 179 feed the electrostatic capacitances detected at the intersectionpoints of the corresponding rows to the detection data generation unit176. Detection data is, for example, digital data generated by digitalsampling of voltage values corresponding to the electrostaticcapacitances at the intersection points. The detection data is providedto the detection unit 171. The detection unit 171 controls the sensorunit 174 to perform the detection periodically at detection positions(the intersection points where the capacitive sensor elements 180 areformed) of the operation panel 177.

Control of the capacitive touch panel 215 includes controlling timingsof applying voltages to the electrodes Ey 179 by the driving unit 175and includes controlling timings of reading detection data from theelectrodes Ey 179 by the detection data generation unit 176.

The determination unit 172 stores amounts of change in the electrostaticcapacitances at the detection positions on the operation panel 177 inthe storage unit 173 based on the detection data received from thesensor unit 174 via the detection unit 171. The storage unit 173 storesreference value data indicating coordinates and electrostaticcapacitances of the sensor elements 180 and threshold values including athreshold value T1 for pointer display and a threshold value T2 forcontact input.

FIG. 11 is a flowchart illustrating a user input detection operation bythe operation unit 200 according to the first exemplary embodiment. Instep S301, after activation, the operation unit 200 starts infraredsensor operations for contactless input in order to receive contactlessinput. Specifically, a microcomputer 202 on the control board 201operates the display device 209 and the contactless touch panel 210under control by the touch panel control unit 204. Next, in step S302,the microcomputer 202 updates a liquid crystal display (LCD) bydisplaying image data generated by the image processing unit 203 on thedisplay device 209 via the display device I/F 206 based on data receivedfrom the controller unit 100 via the controller I/F 205.

Next, in step S303, the microcomputer 202 determines whether user inputis received. The microcomputer 202 determines the presence/absence ofreceived user input by determining whether coordinate information isreceived from the contactless touch panel 210 via the contactless touchpanel I/F 207 under control of the touch panel control unit 204. In acase where the coordinate information is received (YES in step S303),the processing proceeds to step S304. In a case where no coordinateinformation is received (NO in step S303), the processing proceeds tostep S305. In a case where the microcomputer 202 detects the coordinateinformation, the coordinate information of the contactless touch panel210 is cleared, and if next coordinate information is detected, the newcoordinate information is stored. Clearing of the coordinate informationis not limited to the above-described case, and in a case where themicrocomputer 202 detects an update with new coordinate information, aflag can be provided to indicate the presence/absence of information byturning the flag on/off. In the present exemplary embodiment, thecoordinate information is cleared.

Returning to FIG. 8 , decreased intensities of light received by thelight receiving elements 169-13 to 169-18 and the light receivingelements 159-8 to 159-11, since infrared light from the light emittingelements 158 and 168 is blocked, indicate the presence of a user'sfinger at an area of coordinate values where the intensities of lighthave decreased.

In a case where the coordinate values have a range, the determinationunit 213 transforms the coordinate values into appropriate values forthe area pointed to by a user's finger and stores the transformed valuesin the storage unit 214. The microcomputer 202 transmits the coordinatesof the user input to the microcomputer 202 via the contactless touchpanel I/F 207 and ends the operation. In a case where a plurality ofregions each with a decrease in received light intensity is detected, anotification of coordinates of each region is transmitted. In thepresent exemplary embodiment, a maximum number of regions is set to two,and in a case where three or more regions are detected, it is determinedthat no input is received.

The coordinate information to be notified can include all thecoordinates at which an intensity in received light has decreased orvalues transformed by a predetermined transformation method, as long asthe coordinates indicating the user input can be notified. In thepresent exemplary embodiment, the determination unit 213 calculates avalue approximate to an average value of coordinates 8 to 11 of thelight receiving element 159, which is 9, and a value approximate to anaverage value of coordinates 13 to 18 of the light receiving element169, which is 15, as appropriate values for one region, and obtains thecoordinates (9, 15). The determination unit 213 stores the coordinatesin the storage unit 214. The microcomputer 202 detects the coordinatesstored in the storage unit 214 by polling. In a case where themicrocomputer 202 detects the coordinates, the detected coordinates areerased from the storage unit 214 to enable notification of nextcoordinates.

Turning back to FIG. 11 , in step S304, the coordinates (9, 15) aretransmitted to the controller unit 100 via the controller I/F 205. Thecontroller unit 100 determines the user input on the “operation key (3)”based on the coordinates (9, 15). The processing then ends.

In step S305, the microcomputer 202 transmits the notification “nocoordinates data”, and the process ends. The operation unit 200 repeatsthe user input detection operation while the image forming apparatus 10operates.

Next, a user input confirmation operation of the controller unit 100according to the first exemplary embodiment will be described below withreference to the flowchart in FIG. 12 .

In step S311, the controller unit 100 transmits LCD data for screendisplay on the display device 209 of the operation unit 200. The CPU 105of the controller unit 100 transmits the LCD data to the imageprocessing unit 203 of the operation unit 200 via the operation unit I/F199. The LCD data corresponding to an entire screen can be transmittedper unit time, or only difference data corresponding to a changed areacan be transmitted. In the present exemplary embodiment, the LCD datacorresponding to an entire screen is transmitted.

Next, in step S312, it is determined whether user input is received. TheCPU 105 determines whether coordinate information indicating the userinput detected by the operation unit 200 is received from the operationunit I/F 199. Any method for detecting the presence/absence ofcoordinate information by which the CPU 105 can recognize thepresence/absence can be used, such as interrupt detection or pollingdetection. In the present exemplary embodiment, the CPU 105 uses thepolling detection. The coordinate information is information transmittedfrom the microcomputer 202 of the operation unit 200 via the controllerI/F 205 in step S304 or S305 and stored in the storage 502, and the CPU105 detects the stored information by polling.

In a case where it is determined in step S312 that user input isreceived (YES in step S312), in step S313, the CPU 105 holds theoperation input obtained from the coordinate information stored in thestorage 502. In step S314, a pointer indicating that the operation inputis successfully recognized is displayed.

The pointer display can be any display that notifies the user of therecognition of operation input by, for example, placing an arrow mark inthe coordinates area, changing the color of the operation key, reversingblack and white, etc. In the present exemplary embodiment, black andwhite are reversed. At this time, a message prompting the user toperform confirmation input can be displayed. The message prompting theconfirmation input can be, for example, “please perform confirmationinput with the pointer displayed on a key to be selected”.

In step S315, the CPU 105 determines whether the confirmation input isreceived. The confirmation input refers to input on an auxiliary inputkey displayed in an area corresponding to coordinates (1, 1) to (2, 3)in FIG. 8 . The confirmation input is notified as coordinate informationabout a second region. In a case where two or more pieces of coordinateinformation are stored in the storage 502, whether the coordinates arewithin a confirmation input coordinate range is determined. In otherwords, whether the coordinates of a confirm key are received in a statewhere the coordinates of the operation key are held is determined.

In a case where coordinate information within the confirmation inputcoordinate range is received, the CPU 105 determines that theconfirmation input is received (YES in step S315), and the processingproceeds to step S317. In a case where no coordinate information withinthe confirmation input coordinate range is received (NO in step S315),the processing proceeds to step S316.

In step S317, the CPU 105 confirms the held operation input.Confirmation of the operation input is equivalent to a touch in contactinput. Specific operations of confirming the operation input will bedescribed below with reference to FIGS. 13A to 13L.

In step S318, the CPU 105 determines that the operation input of the LCDdata is confirmed, and the CPU 105 updates the LCD data for which thepointer is displayed in step S314 by including the operation inputconfirmation in the LCD data. The processing then ends.

In step S316, since no confirmation input is received in step S315, theCPU 105 determines the LCD data for which the pointer is displayed instep S314 as updated data. The processing then ends.

In a case where no user input is received in step S312 (NO in stepS312), the processing proceeds to step S320. In step S320, the CPU 105stops the display of the pointer. In the present exemplary embodiment,the reversed black and white are restored. In step S321, the LCD dataafter the restoration of the reversed black and white due to the absenceof the pointer display is prepared as updated data. The processing thenends. The prepared LCD data is transmitted to the operation unit 200next time step S311 is performed.

Destination input operations in transmitting a facsimile according tothe first exemplary embodiment will now be described with reference toFIGS. 13A to 13L, along with specific steps in FIG. 11 and FIG. 12 .FIG. 13A illustrates a facsimile destination input screen. A destinationinput area is in an upper area of the screen, and a virtual keyboard(input object) for data input is in a central area of the screen. Thescreen also includes a confirmation object (hereinafter, a “confirm”key) for initiating a data input confirmation instruction, a destinationset object (hereinafter, a “set destination” key) for initiating adestination set instruction, and a cancel object (hereinafter, “cancel”key) for initiating a cancel instruction. A state where a user's fingeris placed over an “A” key and the letter “A” on a black background isdisplayed in a destination field is illustrated.

In a case where a contactless input detection operation is started, theoperation unit 200 receives intensities of light received by the lightreceiving elements 159 and 169 by the detection unit 212 in step S303and notifies the determination unit 213 of information about theintensities of the received light. The determination unit 213 calculatescoordinates of the user's finger (object) based on the information aboutthe area with a decreased intensity of received light. In the presentexemplary embodiment, the coordinates are (10, 6). The determinationunit 213 stores the coordinates (10, 6) in the storage unit 214 andnotifies the microcomputer 202 of the coordinate information. Themicrocomputer 202 transmits the coordinates (10, 6) to the storage 502of the controller unit 100 in step S304.

The CPU 105 of the controller unit 100 detects the presence of thecoordinate information from the storage 502 in step S312. The CPU 105determines that the coordinates (10, 6) correspond to the “A” key fromthe LCD data, and detects that confirmation input is absent in stepsS313 to S315. The CPU 105 adds a black-background letter “A”, whichindicates that the operation input is recognized, to the LCD datadestination input area and updates the LCD data in step S316.

The updated LCD data is transmitted to the operation unit 200 in stepS311 in next input detection and received by the operation unit 200 instep S302. The display on the display device 209 is updated, and theblack-background letter “A” is displayed in the destination input area.

FIG. 13B illustrates a state where a finger of the user's other hand isplaced over the confirm key at the lower left. This indicates that theuser performs confirmation input using a “confirm (5)” key at the lowerleft.

The detection unit 212 of the operation unit 200 receives intensities oflight received by the light receiving elements 159 and 169 in step S303and notifies the determination unit 213 of information about theintensities of the received light. The determination unit 213 calculatescoordinates of the user's finger based on the information about the areawith a decreased intensity of received light. In the present exemplaryembodiment, the coordinates are (10, 6) and (2, 2). The determinationunit 213 stores the coordinates (10, 6) and (2, 2) in the storage unit214 and notifies the microcomputer 202 of the coordinate information.The microcomputer 202 transmits the coordinates (10, 6) and (2, 2) tothe storage 502 of the controller unit 100 in step S304.

The CPU 105 of the controller unit 100 detects the presence of thecoordinate information from the storage 502 in step S312. The CPU 105determines that the coordinates (10, 6) correspond to the “A” key fromthe LCD data, and detects that the coordinates (2, 2) correspond to the“confirm (5)” key in the steps S313 to S315. The CPU 105 determines thecoordinates (10, 6) of the “A” key as being input confirmed, changes the“A” key of LCD data to a white-background letter “A”, and updates theLCD data with a cursor moved to prepare for next input in steps S317 andS318. As in FIG. 13A, the updated LCD data is transmitted to theoperation unit 200 in step S311 in next input detection and received bythe operation unit 200 in step S302. The display on the display device209 is updated, and the white-background letter “A” and the cursor movedto prepare for next input are displayed in the destination input area.

FIG. 13C illustrates a state where the user cancels input using theconfirm key at the lower left while maintains input using the “A” key.The detection unit 212 of the operation unit 200 receives intensities oflight received by the light receiving elements 159 and 169 in step S303and notifies the determination unit 213 of information about theintensities of the received light. The determination unit 213 calculatescoordinates of the user's finger based on the information about the areawith a decreased intensity of received light. In the present exemplaryembodiment, the coordinates are (10, 6).

The determination unit 213 stores the coordinates (10, 6) in the storageunit 214 and notifies the microcomputer 202 of the coordinateinformation. The microcomputer 202 transmits the coordinates (10, 6) tothe storage 502 of the controller unit 100 in step S304.

The CPU 105 of the controller unit 100 detects the presence of thecoordinate information from the storage 502 in step S312. The CPU 105determines that the coordinates (10, 6) correspond to the “A” key fromthe LCD data, and detects that confirmation input is absent in stepsS313 to S315. The CPU 105 adds a black-background letter “A”, whichindicates that the operation input is recognized, to the LCD datadestination input area to display the white-background letter “A” andthe black-background letter “A” and updates the LCD data in step S316.As in FIGS. 13A and 13B, the updated LCD data is transmitted to theoperation unit 200 in step S311 in next input detection and received bythe operation unit 200 in step S302. The display on the display device209 is updated, and the white-background letter “A” and theblack-background letter “A” are displayed in the destination input area.

FIG. 13D illustrates a state where a finger of the user's other hand isplaced over the confirm key at the lower left as in FIG. 13B. Thisindicates that the user performs confirmation input using the “confirm(5)” key at the lower left.

The detection unit 212 of the operation unit 200 receives intensities oflight received by the light receiving elements 159 and 169 in step S303and notifies the determination unit 213 of information about theintensities of the received light. The determination unit 213 calculatescoordinates of the user's finger based on the information about the areawith a decreased intensity of received light. In the present exemplaryembodiment, the coordinates are (10, 6) and (2, 2). The determinationunit 213 stores the coordinates (10, 6) and (2, 2) in the storage unit214 and notifies the microcomputer 202 of the coordinate information.The microcomputer 202 transmits the coordinates (10, 6) and (2, 2) tothe storage 502 of the controller unit 100 in step S304. The CPU 105 ofthe controller unit 100 detects the presence of the coordinateinformation from the storage 502 in step S312. The CPU 105 determinesthat the coordinates (10, 6) correspond to the “A” key from the LCDdata, and detects that the coordinates (2, 2) correspond to the “confirm(5)” key in steps S313 to S315. The CPU 105 determines the coordinates(10, 6) of the “A” key as being input confirmed, changes theblack-background letter “A” of LCD data to a white-background letter“A”, and updates the LCD data with a cursor moved to prepare for nextinput in steps S317 and S318. As in FIGS. 13A to 13C, the updated LCDdata is transmitted to the operation unit 200 in step S311 in next inputdetection and received by the operation unit 200 in step S302. Thedisplay on the display device 209 is updated, and the white-backgroundletters “AA” and the cursor moved to prepare for next input aredisplayed in the destination input area.

FIG. 13E illustrates a state where the input up to “AAA012” is confirmedby the user who has continued the input of the facsimile destination.The user selects the “2” key using the user's right hand and the confirmkey at the lower left using the user's left hand, and the CPU 105 of thecontroller unit 100 updates the LCD data with the “2” key input of thedestination “AAA012” confirmed by the “confirm (5)” key input. As inFIGS. 13A to 13D, the updated LCD data is transmitted to the operationunit 200 in step S311 in next input detection and received by theoperation unit 200 in step S302. The display on the display device 209is updated, and the white-background letters “AAA012” and the cursormoved to prepare for next input are displayed in the destination inputarea.

FIG. 13F illustrates a state where the user cancels input of the confirmkey at the lower left and selects a “3” key. The detection unit 212 ofthe operation unit 200 receives intensities of light received by thelight receiving elements 159 and 169 in step S303 and notifies thedetermination unit 213 of information about the intensities of thereceived light. The determination unit 213 calculates coordinates of theuser's finger based on the information about the area with a decreasedintensity of received light. In the present exemplary embodiment, thecoordinates are (14, 15). The determination unit 213 stores thecoordinates (14, 15) in the storage unit 214 and notifies themicrocomputer 202 of the coordinate information. The microcomputer 202transmits the coordinates (14, 15) to the storage 502 of the controllerunit 100 in step S304.

The CPU 105 of the controller unit 100 detects the presence of thecoordinate information from the storage 502 in step S312. The CPU 105determines that the coordinates (14, 15) correspond to the “3” key fromthe LCD data, and detects that confirmation input is absent in stepsS313 to S315. The CPU 105 adds a black-background letter “3”, whichindicates that the operation input is recognized, to the LCD datadestination input area to display the white-background letters “AAA012”and the black-background letter “3” and updates the LCD data in stepS316.

As in FIGS. 13A to 13E, the updated LCD data is transmitted to theoperation unit 200 in step S311 in next input detection and received bythe operation unit 200 in step S302. The display on the display device209 is updated, and the white-background letters “AAA012” and theblack-background letter “3” are displayed in the destination input area.

FIG. 13G illustrates a state where a finger of the user's other hand isplaced over the confirm key at the lower left. This indicates that theuser performs confirmation input using the “confirm (5)” key at thelower left. The detection unit 212 of the operation unit 200 receivesintensities of light received by the light receiving elements 159 and169 in step S303 and notifies the determination unit 213 of informationabout the intensities of the received light. The determination unit 213calculates coordinates of the user's finger based on the informationabout the area with a decreased intensity of received light. In thepresent exemplary embodiment, the coordinates are (14, 15) and (2, 2).

The determination unit 213 stores the coordinates (14, 15) and (2, 2) inthe storage unit 214 and notifies the microcomputer 202 of thecoordinate information. The microcomputer 202 transmits the coordinates(14, 15) and (2, 2) to the storage 502 of the controller unit 100 instep S304. The CPU 105 of the controller unit 100 detects the presenceof the coordinate information from the storage 502 in step S312. The CPU105 determines that the coordinates (14, 15) correspond to the “3” keyfrom the LCD data, and detects that the coordinates (2, 2) correspond tothe “confirm (5)” key in steps S313 to S315. The CPU 105 determines thecoordinates (14, 15) of the “3” key as being input confirmed, changesthe letter “3” to a white-background letter “3”, and updates the LCDdata with the cursor moved to prepare for next input in steps S317 andS318. As in FIGS. 13A to 13F, the updated LCD data is transmitted to theoperation unit 200 in step S311 in next input detection. The updated LCDdata is received by the operation unit 200 in step S302. The display onthe display device 209 is updated, and the white-background letters“AAA0123” and the cursor moved to prepare for next input are displayedin the destination input area.

FIG. 13H illustrates a state where the user performs input to set thedestination. The user cancels input of the confirm key at the lower leftand selects a “set destination” key.

The detection unit 212 of the operation unit 200 receives intensities oflight received by the light receiving elements 159 and 169 in step S303and notifies the determination unit 213 of information about theintensities of the received light. The determination unit 213 calculatescoordinates of the user's finger based on the information about the areawith a decreased intensity of received light. In the present exemplaryembodiment, the coordinates are (2, 15). The determination unit 213stores the coordinates (2, 15) in the storage unit 214 and notifies themicrocomputer 202 of the coordinate information. The microcomputer 202transmits the coordinates (2, 15) to the storage 502 of the controllerunit 100 in step S304. The CPU 105 of the controller unit 100 detectsthe presence of the coordinate information from the storage 502 in stepS312. The CPU 105 determines that the coordinates (2, 15) correspond tothe “set destination” key from the LCD data, and detects thatconfirmation input is absent in steps S313 to S315.

The CPU 105 reverses black and white of the “set destination” key, whichindicates that the operation input is recognized, and updates the LCDdata in step S316. As in FIGS. 13A to 13F, the updated LCD data istransmitted to the operation unit 200 in step S311 in next inputdetection and received by the operation unit 200 in step S302. Thedisplay on the display device 209 is updated, and the “set destination”key is displayed with black and white reversed.

FIG. 13I illustrates a state where a finger of the user's other hand inFIG. 13H is placed over the confirm key at the lower left. Thisindicates that the user performs confirmation input using the “confirm(5)” key at the lower left.

The detection unit 212 of the operation unit 200 receives intensities oflight received by the light receiving elements 159 and 169 in step S303and notifies the determination unit 213 of information about theintensities of the received light. The determination unit 213 calculatescoordinates of the user's finger based on the information about the areawith a decreased intensity of received light. In the present exemplaryembodiment, the coordinates are (2, 15) and (2, 2). The determinationunit 213 stores the coordinates (2, 15) and (2, 2) in the storage unit214 and notifies the microcomputer 202 of the coordinate information.The microcomputer 202 transmits the coordinates (2, 15) and (2, 2) tothe storage 502 of the controller unit 100 in step S304. The CPU 105 ofthe controller unit 100 detects the presence of the coordinateinformation from the storage 502 in step S312. The CPU 105 determinesthat the coordinates (2, 15) correspond to the “set destination” keyfrom the LCD data, and detects that the coordinates (2, 2) correspond tothe “confirm (5)” key in steps S313 to S315.

The CPU 105 determines the coordinates (2, 15) of the “set destination”key as being input confirmed, and updates the LCD data to a screen fortransmitting a facsimile to the input destination AAA0123 in steps S317and S318. As in FIGS. 13A to 13H, the updated LCD data is transmitted tothe operation unit 200 in step S311 in next input detection and receivedby the operation unit 200 in step S302. The display on the displaydevice 209 is updated, and a facsimile transmission screen in FIG. 13Jis displayed. In the present exemplary embodiment, other pieces of inputinformation (e.g., phone number, various settings) necessary fortransitioning to the facsimile transmission screen are input in advanceby a similar method. In a case where other pieces of input informationis missing, an operation of transitioning to a screen for inputting themissing input information is performed.

FIG. 13K illustrates a state where the user selects a “start” key forstarting facsimile transmission on the facsimile transmission screen inFIG. 13J.

The detection unit 212 of the operation unit 200 receives intensities oflight received by the light receiving elements 159 and 169 in step S303and notifies the determination unit 213 of information about theintensities of the received light. The determination unit 213 calculatescoordinates of the user's finger based on the information about the areawith a decreased intensity of received light. In the present exemplaryembodiment, the coordinates are (2, 25). The determination unit 213stores the coordinates (2, 25) in the storage unit 214 and notifies themicrocomputer 202 of the coordinate information. The microcomputer 202transmits the coordinates (2, 25) to the storage 502 of the controllerunit 100 in step S304.

The CPU 105 of the controller unit 100 detects the presence of thecoordinate information from the storage 502 in step S312. The CPU 105determines that the coordinates (2, 25) correspond to the “start” keyfrom the LCD data, and detects that confirmation input is absent insteps S313 to S315. The CPU 105 reverses black and white of the “start”key, which indicates that the operation input is recognized, and updatesthe LCD data in step S316. As in FIGS. 13A to 13J, the updated LCD datais transmitted to the operation unit 200 in step S311 in next inputdetection and received by the operation unit 200 in step S302. Thedisplay on the display device 209 is updated, and the “start” key isdisplayed with black and white reversed.

FIG. 13L illustrates a state where a finger of the user's other hand inFIG. 13K is placed over the confirm key at the lower left. Thisindicates that the user performs confirmation input using the “confirm(5)” key at the lower left.

The detection unit 212 of the operation unit 200 receives intensities oflight received by the light receiving elements 159 and 169 in step S303and notifies the determination unit 213 of information about theintensities of the received light. The determination unit 213 calculatescoordinates of the user's finger based on the information about the areawith a decreased intensity of received light. In the present exemplaryembodiment, the coordinates are (2, 25) and (2, 2). The determinationunit 213 stores the coordinates (2, 25) and (2, 2) in the storage unit214 and notifies the microcomputer 202 of the coordinate information.The microcomputer 202 transmits the coordinates (2, 25) and (2, 2) tothe storage 502 of the controller unit 100 in step S304.

The CPU 105 of the controller unit 100 detects the presence of thecoordinate information from the storage 502 in step S312. The CPU 105determines that the coordinates (2, 25) correspond to the “start” keyfrom the LCD data, and detects that the coordinates (2, 2) correspond tothe confirm (5) key in steps S313 to S315. The CPU 105 determines thecoordinates (2, 25) of the “start” key as being input confirmed andperforms a facsimile transmission operation in steps S317 and S318.

The “start” key is also used in performing a copy operation on a copyscreen. As with the “start” key operation in facsimile transmission, afinger is placed over the “start” key, black and white are reversed, andthen another finger is placed over the “confirm (5)” key to perform thecopy operation.

As described above, in the present exemplary embodiment, input isconfirmed by acquiring the coordinates of the confirm key asconfirmation information in a state where the coordinates of theoperation key are held. With this configuration, unintended multi-inputand input errors, e.g., the processing proceeds to a next operation withthe input being unset, are prevented.

A second exemplary embodiment will be described below. FIG. 14 is anexternal view and FIG. 15 is a top view, illustrating an image formingapparatus 20 according to the second exemplary embodiment. The presentexemplary embodiment is different than the first exemplary embodiment inthat input is confirmed by inputting confirmation information using amicrophone or a foot switch in a state where coordinates of an operationkey are held. Other aspects of the present embodiment are similar tothose of the first exemplary embodiment, and thus redundant descriptionsthereof are omitted.

Each component of the image forming apparatus 20 that is identical to acomponent of the image forming apparatus 10 is indicated with the samereference numeral as that of the corresponding component, and detaileddescriptions of these identical components are omitted herein. In thepresent exemplary embodiment, a component indicated with a differentreference numeral will be described below.

The image forming apparatus 20 includes a microphone 620 as a voicerecognition device, where the microphone 620 receives user voices. Theimage forming apparatus 20 is turned on by a user stepping on a footswitch 610. The microphone 620 and foot switch 610 can be connected tothe image forming apparatus 20 either physically or via networkinterface. FIG. 16 is a block diagram illustrating the image formingapparatus 20. The foot switch 610 and the microphone 620 are connectedto a controller unit 600.

FIG. 17 is a block diagram illustrating the controller unit 600.Components of the controller unit 600 that correspond to components ofthe controller unit 100 have the same reference numerals as those usedfor the controller unit 100. A detailed description of these similarcomponents is omitted herein.

An option control unit 601 controls the microphone 620 and the footswitch 610 under control of the CPU 105. The microphone 620 receives auser voice and transmits the received user voice to the option controlunit 601. The voice content is identified under control of the CPU 105.The foot switch 610 is stepped on by the user to transmit a switch-onstate to the option control unit 601.

FIG. 18 is an operation procedure in a case where contactless input isconfirmed using the foot switch 610. The steps in FIG. 18 are identicalto those in FIG. 12 , with the exception of step S315 of FIG. 12 . StepS315 of the flowchart in FIG. 12 , which illustrate a process ofconfirming user input by the controller unit 100, is replaced with stepS330 in FIG. 18 .

In step S330, a determination is made whether input via the foot switch610 is received. The presence of input via the foot switch 610 indicatesan input confirmation of the operation input. In a case where input viathe foot switch 610 is received (YES in step S330), the CPU 105determines that the operation input is confirmed, and the processingproceeds to step S317. In a case where no input via the foot switch 610is received (NO in step S330), the processing proceeds to step S316.

Next, an operation procedure in a case where contactless input isconfirmed using voice input will be described with reference to FIG. 19. The steps in FIG. 19 are identical to those in FIG. 12 , with theexception of step S315 of FIG. 12 . Step S315 of the flowchart in FIG.12 , which illustrates a process of confirming user input by thecontroller unit 100, is replaced with step S340 in FIG. 19 .

In step S340, a determination is made whether the voice input “confirm”is received from the microphone 620. The voice input “confirm” from themicrophone 620 indicates an input confirmation of the operation input.In a case where the voice input “confirm” is received from themicrophone 620 (YES in step S340), the CPU 105 determines that theoperation input is confirmed, and the processing proceeds to step S317.In a case where no voice input “confirm” is received from the microphone620 (NO in step S340), the processing proceeds to step S316.

As described above, in the present exemplary embodiment, input isconfirmed by acquisition of confirmation information based on “ON” inputvia the foot switch 610 or the voice input “confirm” from the microphone620 in a state where the coordinates of the operation key are held. Withthis configuration, unintended multi-input and input errors, e.g., theprocessing proceeds to a next operation with the input being unset, areprevented.

While various examples and exemplary embodiments have been describedabove, these examples and embodiments are not seen to be limiting.

Embodiment(s) can also be realized by a computer of a system orapparatus that reads out and executes computer executable instructions(e.g., one or more programs) recorded on a storage medium (which mayalso be referred to more fully as a ‘non-transitory computer-readablestorage medium’) to perform the functions of one or more of theabove-described embodiment(s) and/or that includes one or more circuits(e.g., application specific integrated circuit (ASIC)) for performingthe functions of one or more of the above-described embodiment(s), andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s) and/or controlling the one or morecircuits to perform the functions of one or more of the above-describedembodiment(s). The computer may comprise one or more processors (e.g.,central processing unit (CPU), micro processing unit (MPU)) and mayinclude a network of separate computers or separate processors to readout and execute the computer executable instructions. The computerexecutable instructions may be provided to the computer, for example,from a network or the storage medium. The storage medium may include,for example, one or more of a hard disk, a random-access memory (RAM), aread only memory (ROM), a storage of distributed computing systems, anoptical disk (such as a compact disc (CD), digital versatile disc (DVD),or Blu-ray Disc (BD™), a flash memory device, a memory card, and thelike.

While exemplary embodiments have been described, these embodiments arenot seen to be limiting. The scope of the following claims is to beaccorded the broadest interpretation so as to encompass all suchmodifications and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2021-101993, filed Jun. 18, 2021, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An information processing apparatus comprising: adisplay unit configured to display at least one input objectcorresponding to data input; a first detection unit configured to detecta position of an object located in a vertical direction with respect tothe display unit and that interacts with the display unit in acontactless manner, wherein detecting the position results in detectingan instruction associated with the at least one input object that isdisplayed on the display unit and corresponds to the position of theobject; a second detection unit configured to detect confirmation of theinstruction detected by the first detection unit; and an input unitconfigured to input data corresponding to the at least one input objectin a case where the second detection unit detects confirmation of theinstruction associated with the at least one input object in a statewhere the first detection unit detects the instruction associated withthe at least one input object.
 2. The information processing apparatusaccording to claim 1, wherein the display unit further displays aconfirmation object for confirming input via the at least one inputobject, and wherein the input unit inputs the data corresponding to theat least one input object in a case where an instruction associated withthe confirmation object is detected in a state where the first detectionunit detects an instruction associated with the at least one inputobject.
 3. The information processing apparatus according to claim 2,wherein the display unit displays the at least one input object and theconfirmation object on a single screen.
 4. The information processingapparatus according to claim 1, wherein the first detection unit and thesecond detection unit are detection sensors.
 5. The informationprocessing apparatus according to claim 1, wherein the second detectionunit is a foot switch configured to issue a confirmation instruction tothe input unit in a case where the foot switch is stepped on, andwherein the input unit inputs the data corresponding to the at least oneinput object in a case where the input unit receives the confirmationinstruction via the foot switch in a state where the first detectionunit detects an instruction associated with the at least one inputobject.
 6. The information processing apparatus according to claim 1,wherein the second detection unit is a voice recognition deviceconfigured to issue a confirmation instruction to the input unit in acase where the voice recognition device receives a predeterminedinstruction by voice, and wherein the input unit inputs the datacorresponding to the at least one input object in a case where the voicerecognition device receives the predetermined instruction in a statewhere the first detection unit detects an instruction associated withthe at least one input object.
 7. The information processing apparatusaccording to claim 6, wherein the predetermined instruction is theconfirmation instruction.
 8. The information processing apparatusaccording to claim 1, wherein a pointer is displayed on the display unitat a position corresponding to the position of the object in a casewhere the first detection unit detects the object.
 9. The informationprocessing apparatus according to claim 1, wherein, in a case where thefirst detection unit detects the instruction, the display unit displaysa screen to prompt input of a confirmation instruction.
 10. Theinformation processing apparatus according to claim 1, furthercomprising a facsimile, wherein the at least one input object is aninput object for data input relating to input of a destination of thefacsimile.
 11. The information processing apparatus according to claim1, wherein the first detection unit and the second detection unit are aplurality of infrared sensor elements of an infrared contactless touchpanel, where the infrared sensor elements are located in a verticaldirection with respect to the display unit.
 12. The informationprocessing apparatus according to claim 1, wherein the first detectionunit and the second detection unit are included in a capacitivecontactless touch panel configured to detect the position of the objectfrom the display unit based on an electrostatic capacitance generatedbetween the display unit and the object.
 13. A method for controlling aninformation processing apparatus that includes a display unit configuredto display at least one input object corresponding to data input, themethod comprising: detecting a position of an object located in avertical direction with respect to the display unit and that interactswith the display unit in a contactless manner, wherein detecting theposition results in detecting an instruction associated with the atleast one input object that is displayed on the display unit andcorresponds to the position of the object; receiving a confirmationinstruction to confirm the instruction of the at least one input objectin a state where the instruction is detected; and inputting datacorresponding to the at least one input object in a case where theconfirmation instruction is received.